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Publication Title: 

SYSTEM FOR EMBEDDING ADDITIONAL INFORMATION IN VIDEO DATA, 
AND EMBEDDING METHOD 



Abstract: 

It is one object of the present invention to provide a method and system for 
embedding an electronic watermark directly in an MPEG stream. AConstitutionU 

An intra-macroblock of an l-frame or of a P or B-frame is detected in an MPEG 
stream, and following the detection of an intra-macroblock of the l-frame or of the 
P or B-frame, data for one macroblock are extracted from the MPEG stream and 
buffered. An embedding pattern is embedded in the buffered macroblock without 
changing the length of VLC, and the resultant macroblock is returned to the 
MPEG stream. More specifically, a DC factor is extracted from the buffered 
macroblock, and a pseudorandom number is generated to produce an 
embedding pattern. Then, whether the bit length of the DC factor will not be 
changed by embedding the obtained embedded pattern is determined. When the 
bit length of the DC factor not be changed, the embedding pattern is ennbedded 
in the buffered macroblock. When the bit length will be changed, whether 1/2 of 
the embedding pattern can be embedded is determined. If possible, 1/2 of the 
embedded pattern is embedded in the macroblock. 
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(57) Ab.stract 

An intra-block of the I-frantie or P,B-trame is detected from an 
MPEG stream. When it is detected, one macro block of data is taken out 
from the MPEG stream and buffered. The buffered macro block is 
embedded with a pattern so that the length of the VLC may be invariant. 
The macro block embedded with a pattern is returned back to the MPEG 
slream. More specifically, the DC component is detected from the 
buffered macro block and a pseudo-random number is generated in 
order to generate a pattern to be embedded. It is judged whether or not 
the bit length of the DC component is invariant when the generated 
pattern is embedded. If the bit length is invariant, the pattern is 
embedded in the buffered macro block. When the bit length is varied, it 
is judged whether or not half of the pattern can be embedded. If 
possible, the pattern is embedded. 
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m m ^ 

t^. ^ ^ ^ _ ^ ^ (7) If ^ a i6 ^ V ^ X J; t5 ii i6 ^ ^ 

5 m-mm] 

\Z MPEG2 (DliT't^-^^^^^m^t^ VLC (Variable Length Code) 

10 [^ftS^l] 

J?)iZ^ty^(75S^15<htT, >^(IBM ^^)7i^$.^o ilCDtJ 

^l5{i€i^ii50^1--(Elelctornic Watermarking) tfeP^l^tlv dnb^^;!/^ 

XA>K (baseband) -.itJ^jA^. ^(D^JPEG/MPEG^cDE^^^Tint 5 
20 ;/^J:j3 JPEG Joint Photographic Coding Expers 

Groups. MPEG \t Moving Picture Experts Group BSTS^. 

Xit. #I^¥8-159330 (^t[^MS#^JA9-96-044) . !|#!M¥8-273551 i 
|^JA9-96-076) , #[^¥8-348426 (PI JA9-96-090) . !|#I^¥9-0884 93 
25 PIJA9-97-045) . #il¥9-248272(ll JA9-97-156) . JA9-96-0' 
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it. l.Tnfc^-x;^>H-^c7)iii63i^^i5Sl>'^''^^" JPEG/MPEG^ 

iOl^iffll-li^ AC. DCgK^^A^ VLC ^bLTV.^^. iii65i^StI'#H43 
V.T. VLC (D demultiplex, multiplex M^t/icO, t'-yhXhU- 

^3^8-272721 (PI JA9-96-074) -CB.P frameJlfc^t ^WM:^[^i^t^'i^ 
«?r^{^L.TMPEG2H/^>rtJSSi63it^:^j4^niS5nri^^7i^'- 

U ^ I ^ m 315 /j: V ^ - t # C7) Fnl S Ti^' o 

:^^:}o <j: X A ^ t- ^ ^ T o 
*5ZiO^ *5 <fc 15 X X ^ ^#^-r ^ - 1 -e ^ . 
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G ^ (7)mi^^ c7):^^RiJ^ ^> X X A ^ ^ - <^ ^ ^ » 
r^t^^m?E G t'5=^7=-^Tsn«, stmp e gx h u - A^i^ 

^ I-frame^fctiP.B-fraoe > h 7^ □ ^ ^^Hi^fTV.. I-fra 
.e^ t' \t P. B-f ran>e <;:) ^ > h ^ O >y ^ 7!)^^^ ^ tl fc^^ ^ M P E G 

^ - >(7)a*32.^ ^ □ 7 p ^ ^ M p E g CO 7> h u - A {i^r o 
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9yov^<0«ttiSR^>- l-fra.e*fc« P,B-fra.e C0^>h57D.. 
*^.S«l*aT..m.fl«. «PEG2X h U 
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MPEG2 e5= ^T- WDC<75l!ij^fiK^ \zn\.X-7^U-:ru^y^ (Macrobloc 

k) (16x16 pixei)#{4-t?ffifeii*n, mim(nm\z^'Dom,^^^n^ 

(PO. PI. P2. P3)=£^-^o ^i^'S^ P0^iii6i^tri:s^)*li> ^ 
^Dyn<y^(DMfig^<75Y0(DDC5K5^^+lL., Umzm^-IL. Y2.Y 

ce number = i(0 origin). -7 ^ P ^-nu.ber=j r.#J 0 ^ T tl^ 

S^E^pattern ?\t. P = PM(i. J) ^ tl^ . X, MPEG2(7.^^ , 

^(^llDCBJc5^lCAoTV.?>{iti^ -:>ttI<7^subbIock(Yl=^^bYO. YO^ib 
HfrO-7^D^O':/^<7:>Y3)CD)SSB}c^<7)^<7)^, MPEG2 domainT^I^ t 

^^i:$it^DCEg^timi0^A(M(i.j).Y)t/^^. ^:ATfi. iS*63i^^ 

^TXx>yyiOOT> I-frame^^ct± P.B-frame (D^yVy^^^y^^ 
P.B-fra.e y Vy^Uv ^i^^^^ tltc^^-^ 

Xx>y7'200Ttt. 1 -T^D:/a>y^^<;)x-^^A-y77U>^1-^o 
T/7D70.y^l-:^^^t'>vhXhU-A00-^. YO (D DC ^^0^^^ 
Y3 WDC ^^(D^^^^^^m^' ^iCXr«y7 30O75^bXxv 
y400li> DDSKVariable length code)+DDD (dct_dc_si 

ze luiDinanceX^bitfi^ft^b, ^Mtmx.^^^-^^ ^^"^ ^=0- ^' 
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2. 3t3*t LTT^A^fi5cAz:-r^^'§r« 

Len(D(i,j.Y))==Len(D(i.j.Y) + A(M(iJ).Y)) ...-^ U) 
-eWT^ayp-y^CPMCiJ) ^il*52^t^. --"^^ D(iJ.Y){islice. 
num^i. •7//Dyo.yd7_nuin=j, subblock_num=YlCi3^t ^ DC gfiJc^CD 

A(k.Y)liffl*ii^/^^->k^^ H^^t^M<^s£5^^ii^. Len(x) 
it DC^fiK^C7)M7:>^'x(75P#<7)DDSL+DDD<7)bitfiT^^o ^ 3 0 f;:Len(x) =5: 
m^^t. Len(O) = Lend) = DC^fig^^i: LT' 0" T^^ 

fcsubblock=&'rX(i'-l'--^MTS*iibit length7i^*^^^b7tjH.^;^cJ6 
pifigTfe-S>;5^ DCMfiK5^7i^'4'-e^ofc#l^'3MC^^^«ltbit length 

Xx>yr 5007i^bXx>y^700tt. jii6iA^MmTbit*7;)^*^^^rc:46. 

Len(D(iJ.Y))==Len(D(i.j.Y) + A'(M(iJ).Y)) 
Len(D(i.j,Y))==Len(D(i.j.Y) + A"(M(i.j).Y)) .... ^ (2) 

^(DT^nyn^^t^P' M(i,j) Xl±P" M(i.j)<&ffi*iZ.tyc ---^^^ 

A'(k.Y), A"(k,Y)(i^n^nffi«6iA^Ai5'->k(DYH:fett^P'k, P' 
•k{I^Jj^'-r^^(7)Eg5^cDJi^T^^c ^(2)fe?ifc:^n^<tV^^^^ M 

X7^7 7°800-ei-franie^^cti P.B-fraine CD^ > b ^ :/n >y ^ <7^*ITT 
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W^W-r^h^^ ATA#v^-i5'AX<7)r-5'^^tt^O. line buffertCtS 
m^mt^n^o fUy^zmi Line Buffer Tfe^o ^ 100 

T^lt^ofc7^-^^1§tt-r^A'-y7TTS>^o yo>y^300ti I-fran. 
ettzlt P.B-frame (D^ > h ^ "/O >y ^7 ^ffiglT^ l/D ^ 200(7) 
yi^^fj^i^^ MPEG2 video stream^^ttl b , I-frarae^feli P.B- 

frame W^>h7:^P-y^=&^ai-rS. :/P>y/7400l± cci^m^ST 

C<D^P7^400ti, ^IgB^^rx-^A^X-f >^^ai^>v:/(Da 
taHidingTM detection chip)#^cm^iZ.^t^Mffl•r^^^(' 
;1S'5^S'C$>0. yp >y ^SOOTI-frame^^^ail'i^c^. CCI<£:1^aiTS 
^fiTS.^, yp>y/7400H CCIi&^ait?t^'&. CPUtr interrupt$:± 
tf^So |^±(7)^P-y^l007i^byP>y^4003gti> x-^A^7^^>^^ 

;^;lcyp>y^500(i-7^p-/P'y^A'«y7 7T^Sc y p >y ^500l± l 

.2.3) ^^mr^. 7P>y/7700liMSL^^^^ffi^-^^' ^W^P^y 
/7700l±jiJ^iA;^.A^-><DMgLIS((M(iJ))^^^T^S«"C^^" 
[lilT IT^P^P-y^C 2bit7i^'fiJ^^Te>n^o fctAtfMP@ML(72 
0x480)(Dti^U, 720/16*480/16*2 = 2700bit^^4T§o :/P>y^800 
\i DC Huffman tableT$»^. :iTMPEG2(7)DCcD?SKfiE^Whuf fman ta 
bledSO/IEC 31818-2WTableB.12)5:t&i^-r^o :/P <y i^900t± DC huf f 
man lengthjt«^^«-e$.^o M^^^^^Ho^^^ DC^HSbK^cODDSUD 

DDObit&;&t^^oe,?^j:v^7i^ire.75^^mis-r^ii#^fe^o ^y^iooo 

{± DC nmib^^ (re-embedding) ^gT^^. ^P-y^lOOOti. 

>y /7 900Tbi tfi*^^^^ fj: i^^^ , :^ □ -y ^ 700Tt# ^ nfc/^ ^ - > * 
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3^^i^^(7)-7^pya <y bit stream \zmv. t-^ ^'^^^Zlhtl'T^, 
y:-^^^(D^^mto J^±c757D«y ^5007)^ e>:/n^y^ 1010 j^;^^*^ 

t c D ir { c ^ is ^ >v r s n ^ e> n gi5 ^ -e ^ o 

^^^^^(D^mrnv. I- frameTS^'detect^nfcl^, DC(i. j.Y)^ DC 

(Dwm^^ {i,Li<D^mt±.Bmib'^^'^n\zmc). p(M(i. j).Y)§r 

gJ63i^;^^-> (embedding pattern) M(i. j) (7)YlC?Pt-r^ii*3i^ 

ADC(i.j) = DC(i. j.O)-DC(i, j.l) .M(i.j)=0 

DC(i,j.O)-DC(i. j.2) .M(i,j)=l 

DC(i. j. l)-DC(i. j.O) .M(i. j)=2 

DC(i, j,2)-DC(i. j,0) ,M(i.j)=3 

S.2jADC(i.j)/N --> 0 

SjSjADCd. j)/N -> A>0 

ADC(i. j)<7)^i?<i^a=& 

a2 = 2.Sj(ADC(i.j) - <ADC(i. j)»VN 

z =2j2jADC(i. j)/(a*N) 
75^^ fe^thresholdTH^t, T <*zf^bM*ii^^i"Yes". T>=z^^b 
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-r-5>»W£ii5l5'5v'XxA(7)||J|75^'^^nTVi^ (DVD CPTWG DHSG CFP V 

mmmmX'fi^^)V'f"^f}^^CCl = (l,0)(Copy Once)^miii-^nti 
B#, |^fc;^cf:CCI (Extended CCI. £>(TECCI) O^^M^&itlS L> "ECCI 7!£ L 

" (Dm^^mcci^mibih^mm^ff'^t^o "ecci^)D" o^-^fiitw 

"f^^ JVm^y'-^-^^^ 7^510(DSTB (Set Top Box) T^^S^tl. 

m<£ffl^> ^ffi^nfc^-a-. ^O>y^530T ECCI ^ ^ 75^ ^ ¥IJ ^ 
$n-5, idT. ECCl7i^*fei9 (YES) <i:flj^$n/5:i»'&fi:7n >y ^56 
OT^®75^*^±^nS. fcL ECClT^t^ittnt^yP ^^540^ ECClcDJiid 

1. MPEG2 K^'l'>±T(7)U7;i/^<A(7)ECCI S|a6jA^75^* pI^gT^ D > 
^CDl^A-y^y h^t^n^cDVD format to 

2. MPEG2$M§^b7^c^CD^-XA*> K, DA^J^I^fTo fc^<7)7:J-P ^ H 
p^-r> (Analog domain) *^^<7)^aifc "I^'C^-S CI i:, 

3. ^-XA> H'\(Dit^WffiJb3i^*)"Jfg'C^^il i:. 

4. ii#S¥9-088493;^j: <i: WTokenj^Tfa^S 1 7i 5 MifMgS [C *5 ^t ^ 31 ^ - 
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If ^ (D tg ffl 
(1) t'7=:tx-^/()^b®^7 l/-A(7?^tti^fT^^Si:, 

2. HUfat'x^x-^TimP E GCD t'x^x — If ^Jl 1 IBiK 

3. BiJ|B®^7 U-A75mP E G h'x^x-^CD I-frame l^tzit P. B- 
15 frame co-f > h ^'C$)'5)> ft 2 IS«<75 Ao 

4. HtE'^JPif^^Si^'jitJ^S (3) 

(3 a) /iyy 7 'J y'^^ntz 4^mm^<^ DC s^^^^tht^^^t: . 

20 (3 c) MiBt'-v Hfi7:?^'^^<75^^. -f^JPlt^g^/Vy 77 u >^'snfc 

5. BtrlB#m^^Si6jAtj¥S (3) 75^\ ^^\Z 

Od) MBt'>y h:g75^'^^bL/i^-&. #JJDff l / 2 75^Jl4t>iAi6 
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6. mimmm^^ mmimx'omin^msbihP^j-^'^~->-c$)^. 

7. mm^brnm^^ l-7^DyD>y^ ( 1 6 X 1 6 bl^-tJl/) CDM^-^? 

(1) t'x^T-^Tl'^biJ^^ L/-AC7)^m^fT^^IS<!i. 

( 3 ) tdiB/N' 7 7 7 u > ^ $ n fc /mi^^ \zmmm^ ^ ^ ^ m -r 

10 ^^^i:. 

(2) mEM^yiy-i^tmB^^ntzm^. m^mmyiy-i^^^^'b^bm 

(3) BiTlEA'-y 77 U >y^nfc/hfI^IC, h*x^x-i?'7. h U-ACD 

(4) #tjn1ff^waa63A^nAc/hMi^^ t'x:t7^-^(cMr^Pgi: , 

20 o 

9 . MP E GX h 'J-A'\€^^7&^b^a«63itr:^?iTfe-oT. 

(1) MP E GX h U -Aj&^b I-frame ^ fcti P. B-frame O-T > b ^ 

(2) HdaBI-frame tfefi P.B-frame O-T > h 7 ^ n >y ^ 5 tl 
25 fci^-^, HulHMPEGXhU-ATi^bl ■7i7p:/D>y7^(7:'x-^^^ 
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10 (3) niiw^yy 7 'ry^■^ntc^mi$,^^znmmm^^^*^^^lh-r 
1 1. MP E GX h 'J-At|3(0«^S75^L$:^mT^^}*"C$)^T. 

(1) MP E G7. h U-A>0^b I-frame S/ifi P.B-frame O-f > 

15 yuv^(D^tii^n^^mt. 

(2) MIBI-frame^^cti P.B-frame (7)^ > h ^ >y ^ nfc 

ihL. z.n^n-jy7^}y^t^tmt. 

^WT^C iiSlt^ii-r-S^ MP E Gmi^M75^L^ai:^?*o 

(1) x:'jx-iS'*^e)Cc I $:^m-r^¥^<i:v 

(2) CC lijm^^tlfz.m'^s XtlT-^^^'^ECC I ^^mT^^ 

(3) EGG I Til^^m^nfc^-^. xv^^JU • x-iS'(75^S$:i*»it1'^ 
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(4) E CC I ;5^*^ai$nAj:V^ii'&. TV^)^ ' 'f-'^^ECCl^m 

o 

1 4. t'7=:t7^-^'^(7:'#lJn1t^^^ai-rSfci6c07°P^*^A^:^O^ 

(1) tix^x-^75^e>®#7 ^-AW^ffi^ff ^at^i: ^ 

(2) t<rl51iim7l/-A;5^'^iii$n7t^-^. MIBlii^7 

( 3 ) B^flBA* >y 7 7 U nfc/^^^4' (C#j!lPlf ^7i^$:1t{li•r 



wo 00/13414 



PCT>^JP98/05355 



1/5 





Y=0 


Y=l 








+1 


-1 




+1 


0 






0 


0 




-1 


0 






Y=2 


Y=3 







MPEG domain 
A(K,Y) 











+1 


-2 




+1 


-1 


+1 


0 




-1 


+1 






P 



















P ' 


+1 


0 




0 


-1 




0 


0 




P " - 


0 


0 




0 


0 




-1 


0 

































A'(K,Y) 


+1 


-1 




0 


-1 




0 


0 




A"(K,Y) 


0 


0 




+1 


0 




-1 


+1 



















P'o»P'i Po" 

FIG. 1 



wo 00/13414 



PCT/JP98/05355 



2/5 



> ^START j 



100 




200 



X macroblock^J'tD 



No 
embedding 




400 



embed 
''M(id) 



600 ^ 



embed 

^*Ma,j)*""^"M(i,j) 




FIG. 2 



wo 00/13414 



PCT/JP98/05355 



3/5 



Leii(x) 


Variable length code 


dct dc_size_lumi nance 


DCCO^ 


3 


100 


0 


±0 


3 


00 


1 


±1 


4 


01 


2 


±2;5 


6 


101 


3 


±4, ... 


,7 


7 


110 


4 


±8. „. 


,15 


9 


1110 


5 


±16, 


. ,31 


11 


1111 0 


6 


±32, . 


. ,63 


13 


nil 10 


7 


±64, . 


. ,127 


15 


nil no 


8 


±128, 


... ^55 


17 


nil 1110 


9 


±256, 


... ,511 


19 


nil nil 0 


10 


±512, 


... ,1023 


20 


nil nil 1 


11 


±1024, ... ,2047 



FIG. 3 



wo 00/13414 



PCT/JP98/05355 



4/5 



Digital 
Data 



100 



300 



I-frame 
P,B-frame(7) 



CCI 



400 



200 



Line 
Buffer 



Macro 
Block 
Buffer 



500 



600 



1010 



\ Digital 
Data 



1000 1 



DC 

reembedding 

mm. 



900 



DC 

huffman length 



700 



800 



DC 

huffman table 



Chip-1 



Ecci m^^WA 



FIG. 4 



wo 00/13414 



PCT/JP98/05355 




FIG. 5 



\ 



INTERNATIONAL SEARCH REPORT 



[nternational application No. 
PCT/J-P98/05355 



A. CLASSIFICATION OF SUBJECT MATTER 

Int. CI' H04N7/08, H04N7/24, G09C5/00 



According to International Patent Classification (IPC) or to both national classification and IPC 
B. FIELDS SEARCHED 

Minimum documentation searched (classification system followed by classification symbols) 
Int. CI' H04N7/025-7/088, H04N7/24-7/ 65 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 

Jitsuyo Shinan Koho 1940-1996 Jitsuyo Shinan Toroku Koho 1996-1998 

KoJcai Jitsuyo Shinan Koho 1971-1996 

Electronic data base consulted during the international search (name of data base and, where practicablersearch terms used) 

JICST File (JOIS) Denshi Sukashi, Wotaa Maaku, Deeta Haidingu 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



JP, 10-145757, A (NEC Corp.), • 
29 May, 1998 (29. 05. 98) 
& EP, 840513, A2 

V, Darmstaedetr , et al . , A Block Based Watermarking 
Technique for MPEG2 Signals: Optimization and 
Validation on Real Digital TV Distribution Links, 
Lecture Notes in Computer Science, May 199 8, 
Vol. 1425, pages 190 and 206 (Berlin) 



8, 10 



1-14 



I I Further documents are listed in the continuation of Box C Q See patent tamily annex. 



Sp(;c'iul cutcgories uf cited docunienU>: 

document defining the general state of ihe art which is not 

considered to be of particular relevance 

earlier document but published on or after the international filing date 
document which may throw doubts on priority c]aim(s) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified). 

document referring to an oral disclosure, use, exhibition or other 
means 

document published prior to the international filing date but later than 
the priority date claimed 



"T" later document published after the international filing dale or priority 

date and not in conflict with the application but cited to understand 

the principle or theory underlying the invention 
"X" document of particular relevance; ihe claimed invention cannot be 

considered novel or cannot be considered to involve an inventive step 

when the document is taken alone 
" Y" document of particular relevance; the claimed invention catmoi be 

considered to involve an inventive step when the document is 

combined with one or more other such documents, such combination 

being obvious lo a pcrwn skilled in the art 
" &" document member of the same patent family 



Date of the actual completion of the international search 
23 February, 1999 (23. 02. 99) 


Date of mailing of the international search report 
9 March, 1999 ( 09 . 03 . 99) 


Name and mailing address of the ISA/ 

Japanese Patent Office 

Facsimile No. 


Authorized officer 
Telephone No. 



Form PCrr/ISA/210 (second sheet) (July 1992) 



PCT/ J p 9 8/0 5 3 5 5 



A. &mm't^^^<D'^m mwMt^'Am ( i pc) ) 

Int. Cr I-i04N7/0H, H04N7/24, G09C5/00 



m^m-ofzm^^mn mmm'^^^m d pc) ) 

Int. Cl' HO 4N 7/0 2 5 - 7/0 8 8. H 0 4 N 7 / 2 4 - 7 / 6 5 



1 9 4 0 - 1 9 9 6^;^ 
1 9 7 1 - 1 9 9 6¥ 
1996-1 998¥ 



LTt'iTx-:?^-^ {7'~'^^~':^<D^w. mmcm^Lf^mi^) 









X 


JP, 10-145757,, A (B^m^^^^a:) 
29. 5^. 1998 (2 9. 05. 9 8) 
&EP84051 3, A2 


8, 10 


A 


V. Daxmstaedetr, et al, A Block Based Watermarking Technique 
for MPEG2 Signals: Optimization and Validation on Real 
Digital TV Distribution Links, Lecture Notes in Computer 
Science, May 1998, Vol. 1425, pagesl90 and 206(Berlin) 


1-14 



2 3. 0 2. 9 9 



n;^[lit#t^/f C I s A/ J p) 

SS{ig#^ 1 0 0- 8 9 1 5 



Q g 03.99 



miS^^ 03-3581-1 lO 1 



5 C 



9 6 4 8 



3 5 4 3 



^^PCT/ I S A/2 1 0 ( 1 9 9 8¥7^) 



